The present invention relates to the field of monitoring a driver of a vehicle. In particular, different senses of the driver may be monitored to determine the attention of the driver to the driving activity. Based on the attention of the driver, warning systems and driving assistant systems may be adapted so that the vehicle may drive autonomously as long as possible before providing a signal to the driver to take-over the driving responsibility.
Due to the ongoing development of sensors, monitoring and analyzing the inside and the outside of a vehicle may be possible. For example, cameras may be used to monitor a driver's gaze and field of view and may provide a warning if the driver's gaze might not be oriented in a forward direction.
In particular, US 2008/0061958 A1 discloses an active safety system for a vehicle including an external object sensor system arranged on a host vehicle. The sensor system may be designated to sense objects external to the host vehicle and may generate input data relating to the objects, wherein the input data may include an object position. Additionally, a threat indicator may be included to assign a threat level to each external object detected by the sensor system.
Accordingly, there may be a need for systems which efficiently improve monitoring a driver of a vehicle. In particular, there may be a need for a system which monitors the attention of the driver to the driving activity and may support the driver to take-over driving responsibility when the vehicle might not continue driving autonomously.
This need may be met by a method and system for monitoring a driver of a vehicle in accordance with embodiments of the invention.
According to a first aspect of the invention, there is provided a method for monitoring a driver of a vehicle. The method receives a state of the driver and one or more objects of a real environment model of the vehicle. One or more objects of an envisioned environment model of the driver are generated based on the one or more objects of the real environment model of the vehicle and the state of the driver, wherein, if the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model. And, the one or more objects of the envisioned environment model are updated based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model. This may provide the advantage that the envisioned environment model of the driver may efficiently predict the movement of objects even if the driver does not receive any new information via at least one sense of his senses. Accordingly, a more precise model of the environment may be created from a driver's point of view. More specifically, the environment of the driver may be monitored more efficiently based on the state of the driver.
According to an embodiment of the invention, the generating of the one or more objects of the envisioned environment model may further comprise that a corresponding object of the particular object is found in the envisioned environment model. Next it may be determined whether the driver of the vehicle has received information about the particular object via a subset of senses of the driver, wherein the subset of senses is determined based on the state of the driver. Further, the corresponding object of the envisioned environment model may be updated with data of the particular object based on the subset of senses, if the corresponding object exists in the envisioned environment model and the driver has received information about the particular object via a subset of senses. This may provide the advantage that any sense of the driver may be used to receive information. In particular, only the information of an object of the real information model related to a particular sense or a subset of senses may be included in the envisioned environment model. Accordingly, the envisioned environment model may be efficiently created based on data from different senses.
According to a further embodiment of the invention, the method may further comprise that a deviation factor may be computed between the particular object of the real environment model and the corresponding object of the envisioned environment model. This may provide the advantage that a deviation of the driver's environment and the real environment may be determined at any time by a simple comparison of the envisioned environment model and the real environment model. Thus, a fast response to any deviations may be enabled.
According to a further embodiment of the invention, the deviation factor may comprise a deviation of at least one of position, speed, and acceleration between the particular object of the real environment model and the corresponding object of the envisioned environment model. By comparing each pair of objects of the real environment model and the envisioned environment model, the deviation may be efficiently computed.
According to a further embodiment of the invention, the deviation of the position may comprise a weighted deviation of the position, wherein the deviation of the speed may comprise a weighted deviation of the speed, and/or wherein the deviation of the acceleration may comprise a weighted deviation of the acceleration. By using weights when computing the deviation, the computation of the deviation factor may be efficiently adapted to different situations.
According to a further embodiment of the invention, the method may further comprise that a situation awareness factor may be calculated based on the deviation factor for the particular object of the real environment model and the corresponding object of the envisioned environment model. By calculation of the situation awareness factor, an indicator may be efficiently calculated which takes into account any deviation of some or all objects included in the environment models.
According to a further embodiment of the invention, the method may further comprise that calculating a take-over ability time based on a function of the situation awareness factor. For example, the take-over ability time may be calculated based on the reverse of the situation awareness factor. By calculating the takeover-ability time based on the situation awareness factor, the estimated time for taking over the vehicle by the driver is related to the awareness of the driver regarding the actual situation around the vehicle.
According to a further embodiment of the invention, the method may further comprise that the calculated situation awareness factor and/or the calculated take-over ability time may be provided to at least one of a warning system and a driving assistant system of the vehicle. Advantageously, the warning system and/or the driving assistant system may be enabled to efficiently adapt their functionalities to the detected situation awareness factor and/or the take-over ability time.
According to a further aspect of the invention there is provided a system for monitoring a driver of a vehicle. The system includes a processor, a memory, and instructions stored within the memory, wherein the instructions, when executed on the processor, cause the system to: receive a state of the driver, receive one or more objects of a real environment model of the vehicle, generate one or more objects of an envisioned environment model of the driver based on the one or more objects of the real environment model of the vehicle and the state of the driver. If the state of the driver indicates that at least one sense of the driver sensed a particular object of the real environment model, the particular object of the real environment model is added to the envisioned environment model. And, the one or more objects of the envisioned environment model are updated based on the state of the driver, wherein the one or more objects of the envisioned environment model are updated by a physics engine if the state of the driver indicates that the driver is prevented from sensing the one or more objects of the real environment model. This may provide the advantage that the envisioned environment model of the driver may efficiently predict the movement of objects even if the driver does not receive any new information via at least one sense of his senses. Accordingly, a more precise model of the environment may be created from a driver's point of view. More specifically, the environment of the driver may be monitored more efficiently based on the state of the driver.
According to a further aspect of the invention there is provided a computer program product for monitoring a driver of a vehicle, the computer program product, when being executed by a data processor, is adapted for controlling and/or for carrying out the method as described above.
As used herein, reference to a computer program or a computer program product is intended to be equivalent to a reference to a program element and/or to a computer readable medium containing instructions for controlling a computer system to coordinate the performance of the above described method.
The computer program may be implemented as computer readable instruction code in any suitable programming language, such as, for example, JAVA, C++, and may be stored on a computer-readable medium (removable disk, volatile or non-volatile memory, embedded memory/processor, etc.). The instruction code is operable to program a computer or any other programmable device to carry out the intended functions. The computer program may be available from a network, such as the World Wide Web, from which it may be downloaded.
The invention may be realized by means of a computer program in the form of software. However, the invention may also be realized by means of one or more specific electronic circuits, i.e. hardware. Furthermore, the invention may also be realized in a hybrid form, i.e. in a combination of software modules and hardware modules.
It has to be noted that embodiments of the invention have been described with reference to different subject matters. In particular, some embodiments have been described with reference to method type claims whereas other embodiments have been described with reference to apparatus type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters, in particular between features of the method type claims and features of the apparatus type claims is considered as to be disclosed with this document.
The aspects defined above and further aspects of the present invention are apparent from the examples of embodiments to be described hereinafter and are explained with reference to the examples of embodiments. The invention will be described in more detail hereinafter with reference to examples of embodiments but to which the invention is not limited.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.